Pigment detection method and electronic device

ABSTRACT

A pigment detection method includes: extracting a first image from a to-be-detected RGB skin image, where the first image is used to represent a body reflection component in the RGB skin image, and the RGB skin image is photographed by a device having an RGB image photographing function; extracting a pigment from an R channel, a B channel, and a G channel of the first image based on a correspondence between a first spectral response curve of the pigment and a second spectral response curve of the device having the RGB image photographing function; and generating a pseudo-color image based on the extracted pigment, and displaying the pseudo-color image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a National Stage of International Application No.PCT/CN2018/106216, filed on Sep. 18, 2018, which claims priority toChinese Patent Application No. 201810776213.1, filed on Jul. 16, 2018,both of which are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

This application relates to the field of terminal technologies, and inparticular, to a pigment detection method and an electronic device.

BACKGROUND

Skin pigment distribution is directly related to skin appearance andmany skin problems. For example, uneven distribution of melanin leads toskin problems such as chloasma, freckles, and sunburn. For anotherexample, a change in hemoglobin content is directly related to acne,sensitive skin, inflammation, and angiotelectasis. Therefore, accuratedetection of skin pigment not only plays an important role in researchand test of effectiveness of skin care products and skin careinstruments, but also provides important guidance for consumers' dailybeauty treatment and skin care.

Currently, skin pigment can be detected using an application on a mobileterminal. A common skin pigment detection manner includes: performingoffline training by using skin images that feature different skin colorsand that are acquired in different scenarios, to obtain a detectionmodel used to separate a skin pigment; and performing detection on atest image using the detection model to separate a skin pigment. Becausemobile detection scenarios vary greatly, it is impossible to acquireskin images in all scenarios for training. Therefore, the detectionmodel is ill-suited to skin pigment detection performed in a scenario ofcomplex variables. As a result, the prior art detection model has poorapplicability.

SUMMARY

This application provides a pigment detection method and an electronicdevice, to resolve a problem of poor applicability of an existing skinpigment detection technology.

According to a first aspect, an embodiment of this application providesa pigment detection method. The method includes: extracting a firstimage from a to-be-detected RGB skin image, where the first image isused to represent a body reflection component in the RGB skin image, andthe RGB skin image is photographed by a device having an RGB imagephotographing function; extracting a pigment from an R channel, a Bchannel, and a G channel of the first image based on a correspondencebetween a first spectral response curve of the pigment and a secondspectral response curve of the device having the RGB image photographingfunction; and generating a pseudo-color image based on the extractedpigment, and displaying the pseudo-color image.

Based on this solution, the first image is extracted from theto-be-detected RGB skin image, where the first image is used torepresent the body reflection component in the RGB skin image, and theRGB skin image is photographed by the device having the RGB imagephotographing function. Further, the pigment is extracted from the firstimage based on the relationship between the first spectral responsecurve of the pigment and the second spectral response curve of thedevice having the RGB image photographing function. In this way, pigmentextraction is performed based on the spectral response relationship, sothat pigments in RGB skin images photographed in different scenarios canbe detected, avoiding a case in the prior art in which pigment detectioncan be performed only after training is performed in advance by usingskin images acquired in different scenarios. Therefore, pigmentdetection based on this solution has relatively good applicability.

Further, to improve fidelity of the first image extracted from the RGBskin image, in a possible implementation, the to-be-detected RGB skinimage is converted into a first Lab image; a body reflection componentis extracted from each of an L channel, an a channel, and a b channel ofthe first Lab image; the body reflection components extracted from the Lchannel, the a channel, and the b channel of the first Lab image arecombined to obtain a second Lab image; and the second Lab image isconverted into an RGB image to obtain the first image.

In this manner, the body reflection components are extracted, so thatcolor-related information (the a channel and the b channel) andcolor-unrelated information (the L channel) can be separated, toseparately process the color-related information, thereby helpingimprove the fidelity of the first image.

In a possible design, the body reflection component of the L channel ofthe first Lab image is a difference between an initial value of the Lchannel of the first Lab image and a surface reflection component of theL channel of the first Lab image; the body reflection component of the achannel of the first Lab image is a difference between an initial valueof the a channel of the first Lab image and a surface reflectioncomponent of the a channel of the first Lab image; the body reflectioncomponent of the b channel of the first Lab image is a differencebetween an initial value of the b channel of the first Lab image and asurface reflection component of the b channel of the first Lab image;and the surface reflection components of the L channel, the a channel,and the b channel of the first Lab image are obtained by separatelyperforming filtering processing on the initial values of the L channel,the a channel, and the b channel of the first Lab image. According tothis design, the surface reflection components of the L channel, the achannel, and the b channel can be filtered out, to accurately extractthe body reflection components of the L channel, the a channel, and theb channel.

In a possible design, to improve pigment detection accuracy, the surfacereflection components of the L channel, the a channel, and the b channelof the first Lab image may be obtained by separately performingbilateral filtering processing on the initial values of the L channel,the a channel, and the b channel of the first Lab image. This design canhelp further retain edge information of the first image, therebyimproving the pigment detection accuracy.

In a possible design, the pigment includes but is not limited to any oneof the following: hemoglobin, melanin, carotene, lipochrome, and bilepigment.

According to a second aspect, an embodiment of this application providesan electronic device, where the electronic device includes a memory, aprocessor, and a display screen. The memory is configured to store aprogram instruction. The processor is configured to read the programinstruction stored in the memory, and perform the following operations:extracting a first image from a to-be-detected RGB skin image, where thefirst image is used to represent a body reflection component in the RGBskin image, and the RGB skin image is photographed by a device having anRGB image photographing function; extracting a pigment from an Rchannel, a B channel, and a G channel of the first image based on acorrespondence between a first spectral response curve of the pigmentand a second spectral response curve of the device having the RGB imagephotographing function; and generating a pseudo-color image based on theextracted pigment, and displaying the pseudo-color image. The displayscreen is configured to display the pseudo-color image.

In a possible design, the processor is specifically configured toperform the following operations: converting the to-be-detected RGB skinimage into a first Lab image; extracting a body reflection componentfrom each of an L channel, an a channel, and a b channel of the firstLab image; combining the body reflection components extracted from the Lchannel, the a channel, and the b channel of the first Lab image toobtain a second Lab image; and converting the second Lab image into anRGB image to obtain the first image.

In a possible design, the body reflection component of the L channel ofthe first Lab image is a difference between an initial value of the Lchannel of the first Lab image and a surface reflection component of theL channel of the first Lab image; the body reflection component of the achannel of the first Lab image is a difference between an initial valueof the a channel of the first Lab image and a surface reflectioncomponent of the a channel of the first Lab image; the body reflectioncomponent of the b channel of the first Lab image is a differencebetween an initial value of the b channel of the first Lab image and asurface reflection component of the b channel of the first Lab image;and the surface reflection components of the L channel, the a channel,and the b channel of the first Lab image are obtained by separatelyperforming filtering processing on the initial values of the L channel,the a channel, and the b channel of the first Lab image.

In a possible design, that the surface reflection components of the Lchannel, the a channel, and the b channel of the first Lab image areobtained by separately performing filtering processing on the initialvalues of the L channel, the a channel, and the b channel of the firstLab image includes: the surface reflection components of the L channel,the a channel, and the b channel of the first Lab image are obtained byseparately performing bilateral filtering processing on the initialvalues of the L channel, the a channel, and the b channel of the firstLab image.

In a possible design, the pigment includes but is not limited to any oneof the following: hemoglobin, melanin, carotene, lipochrome, and bilepigment.

According to a third aspect, an embodiment of this application providesa computer storage medium, where the computer storage medium stores aprogram instruction, and when the program instruction is run on anelectronic device, the electronic device is enabled to perform themethod according to any one of the first aspect in the embodiments ofthis application or the possible designs of the first aspect.

According to a fourth aspect, an embodiment of this application providesa computer program product. When the computer program product is run onan electronic device, the electronic device is enabled to perform themethod according to any one of the first aspect in the embodiments ofthis application or the possible designs of the first aspect.

According to a fifth aspect, an embodiment of this application providesa chip, where the chip is coupled to a memory in an electronic device,and controls the electronic device to perform the method according toany one of the first aspect in the embodiments of this application orthe possible designs of the first aspect.

In addition, for technical effects brought by the second aspect to thefifth aspect, refer to the description in the first aspect. Details arenot described herein again.

It should be noted that the “coupling” in the embodiments of thisapplication means that two components are directly or indirectlycombined with each other.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram of a structure of an electronic device towhich an embodiment of this application is applicable;

FIG. 2 is a schematic diagram of a user interface according to anembodiment of this application;

FIG. 3 is a schematic flowchart of a pigment detection method accordingto an embodiment of this application;

FIG. 4A and FIG. 4B are a schematic diagram of a user interfaceaccording to an embodiment of this application;

FIG. 5A to FIG. 5D are a schematic diagram of a user interface accordingto an embodiment of this application;

FIG. 6 is a schematic diagram of a spectral response curve according toan embodiment of this application;

FIG. 7 is a schematic diagram of a grayscale image corresponding tomelanin according to an embodiment of this application;

FIG. 8 is a schematic diagram of a grayscale image corresponding tohemoglobin according to an embodiment of this application; and

FIG. 9 is a schematic diagram of a structure of an electronic device towhich an embodiment of this application is applicable.

DESCRIPTION OF EMBODIMENTS

The following further describes in detail the embodiments of thisapplication with reference to accompanying drawings.

It should be understood that, in the embodiments of this application,“at least one” means one or more, and “a plurality of” means two ormore. “and/or” describes an association relationship for describingassociated objects and represents that three relationships may exist.For example, A and/or B may represent the following three cases: Only Aexists, both A and B exist, and only B exists. A and B may be in asingular or plural form. The character “/” generally indicates an “or”relationship between the associated objects. “At least one (item) of thefollowing” or a similar expression thereof means any combination ofthese items, including a single item or any combination of a pluralityof items. For example, at least one (item) of a, b, or c may represent:a; b; c; a and b; a and c; b and c; or a, b, and c, where a, b, and ceach may be in a singular or plural form.

The embodiments disclosed in this application may be applied to anelectronic device.

In some embodiments of this application, the electronic device may be aportable electronic device including a function such as a personaldigital assistant function and/or a music player function, for example,a mobile phone, a tablet computer, a wearable device (such as a smartwatch) with a wireless communication function, or a vehicle-mounteddevice. An example embodiment of a portable electronic device includesbut is not limited to a portable electronic device using iOS®, Android®,Microsoft®, or another operating system. The foregoing portableelectronic device may alternatively be a laptop computer (Laptop) havinga touch-sensitive surface (for example, a touch panel), or the like. Itshould also be understood that, in some other embodiments of thisapplication, the foregoing electronic device may alternatively be adesktop computer having a touch-sensitive surface (for example, a touchpanel).

FIG. 1 is an example of a schematic diagram of a structure of anelectronic device.

The electronic device 100 may include a processor 110, an externalmemory interface 120, an internal memory 121, and a universal serial bus(universal serial bus, USB) interface 130, a charging management module140, a power management module 141, a battery 142, an antenna 2, awireless communications module 160, an audio module 170, a loudspeaker170A, a telephone receiver 170B, a microphone 170C, a headset jack 170D,a sensor module 180, a key 190, a motor 191, an indicator 192, a camera193, a display screen 194, and the like. The sensor module 180 includesan ambient light sensor 180L. In addition, the sensor module 180 mayfurther include a pressure sensor 180A, a gyroscope sensor 180B, abarometric pressure sensor 180C, a magnetic sensor 180D, an accelerationsensor 180E, a distance sensor 180F, an optical proximity sensor 180G, afingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K,a bone conduction sensor 180M, and the like.

In some other embodiments, the electronic device 100 in this embodimentof this application may further include an antenna 1, a mobilecommunications module 150, a subscriber identity module (subscriberidentification module, SIM) card interface 195, and the like.

The processor 110 may include one or more processing units. For example,the processor 110 may include an application processor (AP), a modemprocessor, a graphics processing unit (GPU), an image signal processor(ISP), a controller, a memory, a video codec, a digital signal processor(DSP), a baseband processor, a neural-network processing unit (NPU),and/or the like. Different processing units may be independent devices,or may be integrated into one or more processors.

In some embodiments, a memory may further be disposed in the processor110, and is configured to store an instruction and data. For example,the memory in the processor 110 may be a cache memory. The memory maystore an instruction or data that is recently used or cyclically used bythe processor 110. If the processor 110 needs to use the instruction orthe data again, the processor 110 may directly invoke the instruction orthe data from the memory. This avoids repeated access and reduces awaiting time of the processor 110, thereby improving system efficiency.

In some other embodiments, the processor 110 may further include one ormore interfaces. For example, the interface may be the USB interface130. For another example, the interface may alternatively be aninter-integrated circuit (I2C) interface, an inter-integrated circuitsound (I2S) interface, a pulse code modulation (PCM) interface, auniversal asynchronous receiver/transmitter (UART) interface, a mobileindustry processor interface (MIPI), a general purpose input/output(GPIO) interface, a SIM interface, or the like. It can be understoodthat in this embodiment of this application, different modules of theelectronic device 100 may be connected through an interface, so that theelectronic device 100 can implement different functions, for example,photographing and processing. It should be noted that a connectionmanner of the interface in the electronic device 100 is not limited inthis embodiment of this application.

The USB interface 130 is an interface that complies with a USB standardspecification. For example, the USB interface 130 may include a mini USBinterface, a micro USB interface, a USB type C interface, and the like.The USB interface 130 may be configured to connect to a charger tocharge the electronic device 100, may be configured to transmit databetween the electronic device 100 and a peripheral device, or may beconfigured to connect to a headset and play audio by using the headset.The interface may further be configured to connect to another electronicdevice, for example, an augmented reality (AR) device.

The charging management module 140 is configured to receive a charginginput from the charger. The charger may be a wireless charger, or may bea wired charger. In some embodiments of wired charging, the chargingmanagement module 140 may receive a charging input from a wired chargerby using the USB interface 130. In some embodiments of wirelesscharging, the charging management module 140 may receive a wirelesscharging input by using a wireless charging coil of the electronicdevice 100. While charging the battery 142, the charging managementmodule 140 may further supply power to the electronic device by usingthe power management module 141.

The power management module 141 is configured to connect to the battery142, the charging management module 140, and the processor 110. Thepower management module 141 receives an input from the battery 142and/or the charging management module 140, and supplies power to theprocessor 110, the internal memory 121, the external memory 121, thedisplay screen 194, the camera 193, the wireless communications module160, and the like. The power management module 141 may further beconfigured to monitor parameters such as a battery capacity, a quantityof battery cycles, and a battery health status (electric leakage andimpedance). In some other embodiments, the power management module 141may alternatively be disposed in the processor 110. In some otherembodiments, the power management module 141 and the charging managementmodule 140 may alternatively be disposed in a same device.

A wireless communication function of the electronic device 100 may beimplemented by using the antenna 1, the antenna 2, the mobilecommunications module 150, the wireless communications module 160, themodem processor, the baseband processor, and the like.

The antenna 1 and the antenna 2 are configured to transmit and receiveelectromagnetic wave signals. Each antenna in the electronic device 100may be configured to cover one or more communication frequency bands.Different antennas may further be multiplexed to improve antennautilization. For example, the antenna 1 may be multiplexed as a wirelesslocal area network diversity antenna. In some other embodiments, theantenna may be used in combination with a tuning switch.

The mobile communications module 150 may provide a wirelesscommunication solution applied to the electronic device 100, including2G, 3G, 4G, 5G, or the like. The mobile communications module 150 mayinclude at least one filter, a switch, a power amplifier, a low noiseamplifier (LNA), and the like. The mobile communications module 150 mayreceive an electromagnetic wave by using the antenna 1, performprocessing such as filtering and amplification on the receivedelectromagnetic wave, and transmit a processed electromagnetic wave tothe modem processor for demodulation. The mobile communications module150 may further amplify a signal modulated by the modem processor,convert the amplified signal into an electromagnetic wave by using theantenna 1, and radiate the electromagnetic wave through the antenna 1.In some embodiments, at least some function modules in the mobilecommunications module 150 may be disposed in the processor 110. In someembodiments, at least some function modules in the mobile communicationsmodule 150 and at least some modules in the processor 110 may bedisposed in a same device.

The modem processor may include a modulator and a demodulator. Themodulator is configured to modulate a to-be-sent low-frequency basebandsignal into a medium-high-frequency signal. The demodulator isconfigured to demodulate a received electromagnetic wave signal into alow-frequency baseband signal. Then, the demodulator transmits thelow-frequency baseband signal obtained through demodulation to thebaseband processor for processing. After the low-frequency basebandsignal is processed by the baseband processor, a processed low-frequencybaseband signal is transmitted to the application processor. Theapplication processor outputs a sound signal by using an audio device(which is not limited to the loudspeaker 170A and the telephone receiver170B), or displays an image or a video by using the display screen 194.In some embodiments, the modem processor may be an independent device.In some other embodiments, the modem processor may be independent of theprocessor 110, and disposed in a same device as the mobilecommunications module 150 or another function module.

The wireless communications module 160 may provide wirelesscommunication solutions that are applied to the electronic device 100,for example, wireless local area network (WLAN) (such as wirelessfidelity (Wi-Fi) network), Bluetooth (BT), global navigation satellitesystem (GNSS), frequency modulation (FM), near field communication(NFC), and infrared (IR) technologies. The wireless communicationsmodule 160 may be one or more devices integrated into at least onecommunications processing module. The wireless communications module 160receives an electromagnetic wave signal by using the antenna 2, performsfrequency modulation and filtering processing on the electromagneticwave signal, and sends a processed signal to the processor 110. Thewireless communications module 160 may further receive a to-be-sentsignal from the processor 110, perform frequency modulation andamplification on the signal, convert a processed signal into anelectromagnetic wave by using the antenna 2, and radiate theelectromagnetic wave through the antenna 2.

In some embodiments, the antenna 1 of the electronic device 100 iscoupled to the mobile communications module 150, and the antenna 2 iscoupled to the wireless communications module 160, so that theelectronic device 100 may communicate with a network and another deviceby using a wireless communications technology. The wirelesscommunications technology may include global system for mobilecommunications (GSM), general packet radio service (GPRS), code divisionmultiple access (CDMA), wideband code division multiple access (WCDMA),time-division code division multiple access (TD-SCDMA), long termevolution (LTE), BT, GNSS, WLAN, NFC, FM, an IR technology, and/or thelike. The GNSS may include a global positioning system (GPS), a globalnavigation satellite system (GLONASS), a beidou navigation satellitesystem (BDS), a quasi-zenith satellite system (QZSS), and/or a satellitebased augmentation system (SBAS).

The electronic device 100 implements a display function by using theGPU, the display screen 194, the application processor, and the like.The GPU is a microprocessor for image processing, and is connected tothe display screen 194 and the application processor. The GPU isconfigured to perform mathematical and geometric calculation, and isconfigured to perform graphics rendering. The processor 110 may includeone or more GPUs, and executes a program instruction to generate orchange display information.

The display screen 194 is configured to display an image, a video, andthe like. The display screen 194 includes a display panel. The displaypanel may use a liquid crystal display (LCD), an organic light-emittingdiode (OLED), an active-matrix organic light-emitting diode (AMOLED), aflexible light-emitting diode (FLED), a mini LED, a micro LED, a quantumdot light-emitting diode (QLED), and the like. In some embodiments, theelectronic device 100 may include one or N display screens 194, where Nis a positive integer greater than 1.

The electronic device 100 may implement a photographing function byusing the ISP, the camera 193, the video codec, the GPU, the displayscreen 194, the application processor, and the like.

The ISP is configured to process data fed back by the camera 193. Forexample, during photographing, after a shutter is opened, light istransmitted to a photosensitive element of the camera through a lens, anoptical signal is converted into an electrical signal, and thephotosensitive element of the camera transmits the electrical signal tothe ISP for processing, and converts the electrical signal into a visualimage. The ISP may further perform algorithm-based optimization onnoise, luminance, and a skin color of the image. The ISP may furtheroptimize parameters such as exposure and color temperature of aphotographing scenario. In some embodiments, the ISP may be disposed inthe camera 193.

The camera 193 is configured to capture a static image or video. Anoptical image is generated for an object by using a lens and isprojected onto the photosensitive element. The photosensitive elementmay be a charge coupled device (CCD) or a complementarymetal-oxide-semiconductor (CMOS) phototransistor. The photosensitiveelement converts an optical signal into an electrical signal, and thentransmits the electrical signal to the ISP, to convert the electricalsignal into a digital image signal. The ISP outputs the digital imagesignal to the DSP for processing. The DSP converts the digital imagesignal into an image signal in a standard RGB or YUV format or the like.In some embodiments, the electronic device 100 may include one or Ncameras 193, where N is a positive integer greater than 1.

The DSP is configured to process a digital signal. In addition to thedigital image signal, the DSP may further process another digitalsignal. For example, when the electronic device 100 selects a frequency,the DSP is configured to perform Fourier transform on frequency energy,or the like.

The video codec is configured to compress or decompress a digital video.The electronic device 100 can support one or more types of video codecs.In this case, the electronic device 100 can play or record videos in aplurality of encoding formats, for example, a moving picture expertsgroup (MPEG)-1 format, an MPEG-2 format, an MPEG-3 format, and an MPEG-4format.

The NPU is a neural-network (neural-network, NN) computing processor. Byusing a biological neural network structure, for example, by using amode of transmission between human brain neurons, the NPU can rapidlyprocess input information, and can further perform continuousself-learning. Applications such as intelligent cognition of theelectronic device 100, for example, image recognition, facialrecognition, speech recognition, and text understanding, can beimplemented by using the NPU.

The external memory interface 120 may be configured to connect to anexternal memory card (for example, a micro SD card), to extend a storagecapability of the electronic device 100. The external memory cardcommunicates with the processor 110 by using the external memoryinterface 120, to implement a data storage function, for example,storing a music file, a video file, or the like in the external memorycard.

The internal memory 121 may be configured to store computer executableprogram code, where the executable program code includes an instruction.The processor 110 runs the instruction stored in the internal memory121, to perform various function applications of the electronic device100 and data processing. The internal memory 121 may include a programstorage area and a data storage area. The program storage area may storean operating system, an application required by at least one function(for example, an audio playback function or an image playback function),and the like. The data storage area may store data (for example, audiodata and a phone book) created during use of the electronic device 100,and the like. In addition, the internal memory 121 may include ahigh-speed random access memory, and may further include a nonvolatilememory, for example, at least one magnetic disk storage device, a flashmemory device, and a universal flash storage (universal flash storage,UFS).

The electronic device 100 can implement an audio function, for example,music playback or recording, by using the audio module 170, theloudspeaker 170A, the telephone receiver 170B, the microphone 170C, theheadset jack 170D, the application processor, and the like.

The audio module 170 is configured to convert digital audio informationinto an analog audio signal for output, and is also configured toconvert an analog audio input into a digital audio signal. The audiomodule 170 may further be configured to encode and decode an audiosignal. In some embodiments, the audio module 170 may be disposed in theprocessor 110, or some function modules of the audio module 170 aredisposed in the processor 110.

The loudspeaker 170A, also referred to as a “speaker”, is configured toconvert an audio electrical signal into a sound signal. The electronicdevice 100 can be used for listening to music or answering a hands-freecall by using the loudspeaker 170A.

The telephone receiver 170B, also referred to as an “earpiece”, isconfigured to convert an audio electrical signal into a sound signal.When a call or voice information is received on the electronic device100, voice can be heard by putting the telephone receiver 170B near ahuman ear.

The microphone 170C, also referred to as a “voice tube” or a “mike”, isconfigured to convert a sound signal into an electrical signal. Whenmaking a call or sending voice information, a user can make a sound nearthe microphone 170C with the user's mouth to input a sound signal to themicrophone 170C. At least one microphone 170C may be disposed in theelectronic device 100. In some other embodiments, two microphones 170Cmay be disposed in the electronic device 100, and can further implementa noise reduction function in addition to sound signal acquisition. Insome other embodiments, three, four, or more microphones 170 C mayalternatively be disposed in the electronic device 100, to implementsound signal acquisition and noise reduction, further identify a soundsource, and implement a directional recording function and the like.

The headset jack 170D is configured to connect to a wired headset. Theheadset jack 170D may be a USB interface 130, or may be a 3.5-mm openmobile electronic device platform (OMTP) standard interface, a cellulartelecommunications industry association of the USA (CTIA) standardinterface, or the like.

The pressure sensor 180A is configured to sense a pressure signal, andcan convert the pressure signal into an electrical signal. In someembodiments, the pressure sensor 180A may be disposed on the displayscreen 194. There are many types of pressure sensors 180A, for example,a resistive pressure sensor, an inductive pressure sensor, and acapacitive pressure sensor. The capacitive pressure sensor may includeat least two parallel plates including a conducting material. When forceis applied to the pressure sensor 180A, capacitance between electrodeschanges. The electronic device 100 determines pressure intensity basedon the capacitance change. When a touch operation is performed on thedisplay screen 194, the electronic device 100 detects a strength of thetouch operation by using the pressure sensor 180A. The electronic device100 may alternatively obtain a touch position through calculation basedon a signal detected by the pressure sensor 180A. In some embodiments,touch operations that are performed in a same touch position but havedifferent touch operation strengths may be corresponding to differentoperation instructions. For example, when a touch operation with a touchoperation strength less than a first pressure threshold is performed onan icon of an SMS message application, an instruction for viewing ashort message is executed. When a touch operation with a touch operationstrength greater than or equal to the first pressure threshold isperformed on the icon of the SMS message application, an instruction forcreating a new short message is executed.

The gyroscope sensor 180B may be configured to determine a movingposture of the electronic device 100. In some embodiments, angularvelocities of the electronic device 100 relative to three axes (that is,x, y, and z axes) may be determined by using the gyroscope sensor 180B.The gyroscope sensor 180B may be used for image stabilization duringphotographing. For example, when a shutter is pressed, the gyroscopesensor 180B detects an angle at which the electronic device 100 shakes,calculates, based on the angle, a distance for which a lens module needsto compensate, and allows a lens to offset the shake of the electronicdevice 100 through reverse movement, to implement image stabilization.The gyroscope sensor 180B may also be used for navigation and a motionsensing game scenario.

The barometric pressure sensor 180C is configured to measure atmosphericpressure. In some embodiments, the electronic device 100 calculates analtitude by using the atmospheric pressure measured by the barometricpressure sensor 180C, to assist positioning and navigation.

The magnetic sensor 180D includes a Hall effect sensor. The electronicdevice 100 may detect, by using the magnetic sensor 180D, whether a flipleather case is open or closed. In some embodiments, when the electronicdevice 100 is a flip phone, the electronic device 100 may detect, byusing the magnetic sensor 180D, whether a flip cover is open or closed,and further set, based on a detected open/closed state of a leather caseor a detected open/closed state of the flip cover, attributes such asauto-unlocking is implemented when the flip phone is flipped open.

The acceleration sensor 180E may detect magnitude of accelerations ofthe electronic device 100 in various directions (generally three axes);may detect magnitude and a direction of gravity when the electronicdevice 100 is still; and may further be configured to recognize aposture of the electronic device, and applied to screen switchingbetween a landscape mode and a portrait mode, a pedometer, or anotherapplication.

The distance sensor 180F is configured to measure a distance. Theelectronic device 100 may measure a distance by using infrared or laser.In some embodiments, in a photographing scenario, the electronic device100 may measure a distance by using the distance sensor 180F, toimplement fast focusing.

The optical proximity sensor 180G may include, for example, a lightemitting diode (LED) and an optical detector, for example, a photodiode.The light emitting diode may be an infrared light emitting diode. Theelectronic device 100 emits infrared light by using the light emittingdiode. The electronic device 100 detects infrared reflected light froman object nearby by using a photodiode. When sufficient reflected lightis detected, the electronic device 100 can determine that there is anobject near the electronic device 100. When insufficient reflected lightis detected, the electronic device 100 can determine that there is noobject near the electronic device 100. By using the optical proximitysensor 180G, the electronic device 100 may detect that a user holds theelectronic device 100 close to an ear during a call, to automaticallyturn off a screen for power saving. The optical proximity sensor 180Gmay also be used for automatic screen unlocking or locking in a leathercase mode or a pocket mode.

The ambient light sensor 180L is configured to sense ambient lightbrightness. The electronic device 100 may adaptively adjust brightnessof the display screen 194 based on the sensed ambient light brightness.The ambient light sensor 180L may also be configured to automaticallyadjust white balance during photographing. The ambient light sensor 180Lmay also cooperate with the optical proximity sensor 180G to detectwhether the electronic device 100 is in a pocket, to prevent touch bymistake.

The fingerprint sensor 180H is configured to acquire a fingerprint. Byusing a feature of the acquired fingerprint, the electronic device 100can implement unlocking via the fingerprint, access an application lock,perform photographing via the fingerprint, answer a call via thefingerprint, and the like.

The temperature sensor 180J is configured to detect temperature. In someembodiments, the electronic device 100 executes a temperature processingpolicy by using the temperature detected by the temperature sensor 180J.For example, when the temperature reported by the temperature sensor1807 exceeds a threshold, the electronic device 100 reduces performanceof a processor located nearby the temperature sensor 1807, to reducepower consumption to implement thermal protection. In some otherembodiments, when the temperature is less than another threshold, theelectronic device 100 heats the battery 142 to prevent abnormalpower-off of the electronic device 100 resulted from low temperature. Insome other embodiments, when the temperature is less than still anotherthreshold, the electronic device 100 increases an output voltage of thebattery 142 to prevent abnormal power-off resulted from low temperature.

The touch sensor 180K is also referred to as a “touch panel”. The touchsensor 180K may be disposed in the display screen 194, and the touchsensor 180K and the display screen 194 constitute a touchscreen, whichis also referred to as a “touch control screen”. The touch sensor 180Kis configured to detect a touch operation performed on or near the touchsensor 180K. The touch sensor may transmit the detected touch operationto the application processor to determine a type of a touch event. Avisual output related to the touch operation may be provided by usingthe display screen 194. In some other embodiments, the touch sensor 180Kmay alternatively be disposed in a position, different from a positionof the display screen 194, on a surface of the electronic device 100.

The bone conduction sensor 180M may obtain a vibration signal. In someembodiments, the bone conduction sensor 180M may obtain a vibrationsignal of a vibrating bone block of a vocal-cord part of a human body.The bone conduction sensor 180M may also be in contact with pulses of ahuman body to receive blood pressure fluctuating signals. In someembodiments, the bone conduction sensor 180M may also be disposed in aheadset, to combine with the headset into a bone conduction headset. Theaudio module 170 may obtain a voice signal by parsing the vibrationsignal of the vibrating bone block of the vocal-cord part obtained bythe bone conduction sensor 180M, to implement a voice function. Theapplication processor may obtain heart rate information by parsing theblood pressure fluctuating signals obtained by the bone conductionsensor 180M, to implement a heart rate detection function.

The key 190 may include a power key, a volume key, and the like. The key190 may be a mechanical key, or may be a touch key. The electronicdevice 100 may receive a key input, and generate a key signal inputrelated to a user setting and function control of the electronic device100.

The motor 191 may generate a vibration alert. The motor 191 may beconfigured to provide an incoming-call vibration alert and a touchvibration feedback. For example, touch operations performed on differentapplications (for example, photographing and audio playback) may becorresponding to different vibration feedback effects. The motor 191 mayalso be corresponding to different vibration feedback effects for touchoperations performed in different areas of the display screen 194.Different application scenarios (for example, time reminding,information receiving, an alarm clock application, and a gameapplication) may also be corresponding to different vibration feedbackeffects. The touch vibration feedback effect may also be user-defined.

The indicator 192 may be an indicator light, and may be configured toindicate a charging status and a battery level change, or may beconfigured to indicate a message, a missed call, a notification, or thelike.

The SIM card interface 195 is configured to connect to a SIM card. TheSIM card may be inserted into the SIM card interface 195 or removed fromthe SIM card interface 195, to be in contact with or be separated fromthe electronic device 100. The electronic device 100 can support one orN SIM card interfaces, where N is a positive integer greater than 1. TheSIM card interface 195 can support a nano-SIM card, a micro-SIM card, aSIM card, and the like. A plurality of cards may be inserted into a sameSIM card interface 195 at the same time. The plurality of cards may beof a same type or different types. The SIM card interface 195 may alsobe compatible with different types of SIM cards. The SIM card interface195 may also be compatible with an external memory card. The electronicdevice 100 interacts with a network by using a SIM card, to implement acall function, a data communication function, and the like. In someembodiments, the electronic device 100 uses an eSIM, that is, anembedded SIM card. The eSIM card may be embedded in the electronicdevice 100 and cannot be separated from the electronic device 100.

It can be understood that the schematic structure in this embodiment ofthis application does not constitute any specific limitation on theelectronic device 100. In some other embodiments of this application,the electronic device 100 may include components more or fewer thanthose shown in the figure, or some components may be combined, somecomponents may be split, or there may be a different componentarrangement. The components shown in the figure may be implemented byhardware, software, or a combination of software and hardware.

The following describes this embodiment of this application in detail byusing the electronic device 100 as an example.

In addition, it should be understood that applications supported by theelectronic device in this embodiment of this application may include aphotographing application, for example, a camera. In addition, theapplications supported by the electronic device may further include aplurality of other applications, for example, drawing, gaming, phone,video player, music player, photo management, browser, calendar, andclock.

The applications supported by the electronic device in this embodimentof this application may further include an application for skin test.The application for skin test is to detect a facial skin features (forexample, wrinkles, pores, blackheads, color spots, or a red area offacial skin) of a user by using a photographed facial image, and mayprovide a detection result report for the user. For example, thedetection result report may include, but is not limited to, a score foreach feature of the facial skin and comprehensive analysis on the facialskin, and may further display the facial image of the user, and mark acorresponding problem on the facial image based on a detection result ofeach feature. For example, blackheads are marked in a nose area,wrinkles are marked in a forehead area, and color spots are marked in acheek area. It can be understood that the detection result report may bepresented to the user on a user interface. For example, the detectionresult report may be presented on the user interface 200 shown in FIG. 2, and includes a comprehensive score, a skin age, and scores for pores,blackheads, fine lines, color spots, and a red area. In some otherembodiments, the user interface 200 may further include a virtual button201, a virtual button 202, a virtual button 203, a virtual button 204,and a virtual button 205. Using the virtual button 201 as an example, inresponse to an operation performed on the virtual button 201, theelectronic device 100 displays a specific care advice for the pores onthe display screen 194. For functions of the virtual button 202, thevirtual button 203, the virtual button 204, and the virtual button 205,refer to a function of the virtual button 201. Details are not describedherein again.

To make the electronic device more accurately detect the facial skin ofthe user, for example, in the user skin test solution in this embodimentof this application, a photographing condition detection module, animage quality detection module, and a region of interest (ROI) detectionmodule, a skin feature detection module, a result analysis module, andthe like may be integrated into the processor 110. In some embodiments,the photographing condition detection module, the image qualitydetection module, and the region of interest (ROI) detection module, theskin feature detection module, the result analysis module, and the likemay be integrated into the application processor in the processor 110.In some other embodiments, an artificial intelligence (AI) chip isintegrated into the processor 110, and the photographing conditiondetection module, the image quality detection module, and the region ofinterest (ROI) detection module, the skin feature detection module, theresult analysis module, and the like are integrated into the AI chip, toimplement user skin test.

The photographing condition detection module may detect a currentphotographing condition, to guide a user to perform photographing in arequired photographing condition, to ensure that a photographed imagemeets a requirement, thereby ensuring accuracy of skin test performedbased on the image. For example, the required photographing conditionincludes: ambient light is sufficient; there is an appropriate distance(for example, approximately 25 cm) between a human face and theelectronic device; a face is straight; eyes are closed; no glasses areworn; a forehead is not covered by bangs as far as possible; focusing isaccurate; there is no obvious shake; and the like.

After the photographing condition detection module performs detectionsuccessfully, the processor 110 enables intelligent light compensation.For example, when a current photographing condition meets a requirement,the photographing condition detection module determines that thedetection succeeds. Specifically, in this embodiment of thisapplication, the electronic device may use different light compensationmodes (for example, a flash lamp mode or a flashlight mode) to performlight compensation for a face of a user, to meet requirements ofdetecting different facial skin features. After performing lightcompensation for the face of the user, the processor 110 may control thecamera 193 to photograph the face of the user to obtain a facial imageof the face of the user.

The image quality detection module may detect quality of the facialimage, to ensure that the photographed image meets the requirements ofdetecting different facial skin features.

After the image quality detection module finds that the image qualitymeets the requirements, the ROI detection module may determine ato-be-detected ROI from the facial image. For example, an ROI ofblackheads is a small area on a nose.

The skin feature detection module may detect each of facial skinfeatures in the determined ROI, for example, detect wrinkles, pores,blackheads, color spots, a red area, and a degree of oiliness on theskin.

The result analysis module may analyze a detection result of the facialskin features detected by the skin feature detection module, and providea score, a score ranking, and the like of each detection item for eachskin feature.

In addition, in some embodiments, an image preprocessing module mayfurther be integrated into the processor 110. The image preprocessingmodule may perform compression, cropping, and the like on thephotographed facial image, so that the ROI detection module, the skinfeature detection module, and the like perform subsequent processing.

To output a facial image analysis result, output a score of eachdetection item, or the like, the processor 110 may further display adetection report (including areas with a detection result of eachfeature in the facial image, for example, a nose area marked withblackheads, a forehead area marked with wrinkles, a cheek area markedwith color spots; scores of all detection items; and the like) obtainedthrough detection on the display screen 194 for the user to view,thereby improving user experience.

The following describes the embodiments of this application in detailwith reference to the structure of the electronic device shown in FIG. 1.

To adapt to changes of a plurality of detection scenarios and resolve aproblem of poor applicability of an existing skin pigment detectiontechnology, an embodiment of this application provides a pigmentdetection method. In this method, the electronic device 100 can extracta pigment from an RGB skin image. Because a skin pigment (for example,melanin or hemoglobin) has a specific absorption spectrum, after a bodyreflection component is extracted from the skin image, the pigment canbe separated from the body reflection component by using a spectrumanalysis technology. Therefore, relatively good applicability can beachieved by using the pigment detection method for pigment detection.

The pigment detection method provided in this embodiment of thisapplication may be applied to an application that is used for skin testand that is supported by the electronic device 100. For example, asshown in FIG. 5A to FIG. 5D, the display screen 194 of the electronicdevice 100 displays an icon 500 of an application for skin test. Iffinding an operation performed on the icon 500 (for example, theelectronic device finds that a user taps the icon 500), in response tothe operation performed on the icon 500, the electronic device 100displays a user interface 510 of the application for skin test on thedisplay screen 194. The user interface 510 of the application for skintest includes a virtual button 511 (during implementation, the virtualbutton may be named as “test”, “take a photo”, or the like). If findingan operation performed on the virtual button 511 (for example, theelectronic device finds that the user taps the virtual button 511), inresponse to the operation performed on the virtual button 511, theelectronic device 100 performs, according to the pigment detectionmethod provided in this embodiment of this application, pigmentdetection on an area that is in an RGB skin image and in which a pigmentneeds to be detected.

The RGB skin image may be obtained by the electronic device 100 inresponse to the operation on the virtual button 511 by photographing aface of the user by using the camera 193. The camera 193 herein may be afront-facing camera or a rear-facing camera of the electronic device100. Alternatively, the RGB skin image may be an image that is read, bythe electronic device 100 in response to the operation on the virtualbutton 511, from the internal memory 121 or from an external memory byusing the external memory interface 120. In this case, the RGB skinimage may be an RGB skin image that is photographed in advance and thatis stored in the internal memory 121 or the external memory.

For example, the RGB skin image may be an image obtained by theelectronic device by photographing the face of the user by using thecamera 193 (the camera 193 herein may be a front-facing camera or arear-facing camera). After photographing, the electronic device 100stores the obtained RGB skin image in the internal memory 121, and afterfinding the operation performed on the virtual button 511, theelectronic device 100 may read the RGB skin image from the internalmemory 121. In addition, during implementation, an RGB skin image storedin the internal memory 121 may alternatively be an image received by theelectronic device 100 by using the mobile communications module 150and/or the wireless communications module 160.

Further, after the electronic device 100 finds the operation performedon the virtual button 511, the user may alternatively choose whether theelectronic device 100 performs photographing by using the camera 193 toobtain an RGB skin image or the electronic device 100 reads an RGB skinimage from the internal memory 121 or the external memory. For example,after the electronic device 100 finds the operation performed on thevirtual button 511, the display screen 194 displays a photo selectionarea 512. The photo selection area 512 may include prompt informationsuch as “provide a photo” and “obtain a photo from”, to remind the userto select a source of the RGB skin image, and the photo selection area512 may further include a plurality of virtual buttons. Operationscorresponding to the virtual buttons are performed based on operationsperformed on the virtual buttons by the user, to obtain RGB skin imagesin different ways. For example, the virtual button may be a first button513 (a name of the first button 513 may be “camera”, “take a photo”, orthe like) that represents obtaining an RGB skin image in a photographingmanner; or the virtual button may be a second button 514 (a name of thesecond button 514 may be “storage”, “album”, or the like) thatrepresents obtaining an RGB skin image by reading from a memory. Afterfinding an operation performed by the user on the first button 513, inresponse to the operation performed by the user on the first button 513,the electronic device 100 may photograph a facial image of the user byusing the camera 193, and use the facial image as an RGB skin image.After finding an operation performed by the user on the second button514, the electronic device 100 may continue to remind the user to selectan RGB skin image storage path, and read, from the storage path selectedby the user, an image selected by the user as an RGB skin image. Thestorage path may be a default storage path of an “album” of theelectronic device 100. The storage path may include a storage path ofthe internal memory 121, or may include a storage path of the externalmemory. In addition, it should be understood that the display of thephoto selection area 512 may alternatively be triggered in a mannerother than finding, by the electronic device 100, the operationperformed on the virtual button 511. For example, a new virtual functionbutton may be disposed on the user interface 510, to display the photoselection area 512 after the electronic device 100 finds an operationperformed on the new virtual function button.

After the RGB skin image of the user is obtained by using the foregoingmethod, the display screen 194 may display an RGB skin image previewinterface 520, and display the RGB skin image in a preview area 521 ofthe RGB skin image preview interface 520. The electronic device 100 maydetermine an ROI for pigment detection based on the RGB skin image inthe preview area 521. For example, the electronic device 100 mayautomatically select an ROI through positioning analysis based on afacial feature point in the RGB skin image; the user may manually drawan ROI; or the electronic device 100 may provide an area, and the usermay manually adjust the area provided by the electronic device 100 toobtain an ROI. Then, the ROI is used as an area that is in the RGB skinimage and in which a pigment needs to be detected, and is used toperform pigment detection by using the pigment detection method providedin this embodiment of this application.

The following describes in detail how the processor 110 in theelectronic device 100 specifically implements pigment detection on RGBskin images based on the RGB skin images obtained in the foregoingdifferent manners. For a specific method, refer to a schematic flowchartshown in FIG. 3 . The method includes the following steps.

Step 301. The processor 110 extracts a first image from a to-be-detectedRGB skin image.

The first image is used to represent a body reflection component in theRGB skin image, and the RGB skin image is photographed by a devicehaving an RGB image photographing function.

Herein, the RGB skin image may be a skin image of a to-be-detected part,for example, a facial image or a nose area image.

The RGB skin image is obtained by imaging after incident light isreflected through epidermis and absorbed and scattered by the epidermis,dermis, and subcutaneous tissues. The RGB skin image mainly includes twocomponents: a surface reflection component and a body reflectioncomponent.

The surface reflection component is obtained by using energy thatbounces back in a manner similar to mirror reflection when the incidentlight is incident on a skin surface, and may be used to analyze atopology feature of the skin surface, for example, wrinkles, pores,blackheads, and a skin texture.

The body reflection component is obtained by using energy that isreturned after energy of the incident light enters skin and is absorbedand scattered by pigments, collagen, and the like in the epidermis andthe dermis, and may be used to analyze optical properties of thesubcutaneous tissues, for example, distribution of pigments such asmelanin and hemoglobin. For example, the body reflection component maybe extracted from the to-be-detected RGB skin image, to obtain the firstimage.

The device having the RGB image photographing function may be theelectronic device 100, or may be a device connected to the electronicdevice 100, or may be a device not directly connected to the electronicdevice 100. In an example, the indirect connection may be a wirelessconnection, or may be a connection implemented by using another device,to send or transmit, to the electronic device 100, the RGB skin imagephotographed by the device having the RGB image photographing function.For example, the device having the RGB image photographing function isthe electronic device 100. The RGB image photographing function isimplemented by a camera 193 on the electronic device 100. For example,the camera 193 acquires the RGB skin image. For example, when an imageneeds to be acquired, an operation may be performed on a cameraapplication installed in the electronic device 100, to enable the camera193 by using the electronic device 100 to photograph an image. Thecamera application may be an application pre-installed in the electronicdevice 100 at delivery, or may be an application downloaded by a user.It should be noted that the RGB skin image may be an image pre-stored inthe electronic device 100, or may be obtained through in real-timephotographing by enabling the camera 193 by the electronic device 100.

In a specific example, as shown in FIG. 4A and FIG. 4B, a display screen194 of the electronic device 100 displays a home screen. The home screenincludes an icon 400 of a camera application. In addition, the homescreen may further include an email icon, an SMS message icon, a galleryicon, a WeChat icon, and the like. When an image needs to bephotographed, the display screen 194 of the electronic device 100 mayrespond to an operation performed on the icon 400, for example, a touchoperation performed on the icon 400. The display screen 194 displays apreview interface 410, and the preview interface 410 includes a previewarea 411. The preview area 411 may be used to display the RGB skin imageacquired by the camera 193. Using photographing of a facial image as anexample, when the to-be-photographed image is displayed in the previewarea 411, a touch operation may be performed on a virtual button 412, toacquire an RGB skin image.

In another specific example, the electronic device 100 may furtherinclude an application for skin test. As shown in FIG. 5A to FIG. 5D,the display screen 194 of the electronic device 100 displays the icon500 of the application for skin test. In response to an operationperformed on the icon 500, the electronic device 100 displays the userinterface 510 of the application for skin test on the display screen194. The user interface 510 includes the virtual button 511. If theelectronic device 100 finds an operation performed on the virtual button511, in response to the operation performed on the virtual button 511,the display screen 194 displays a preview interface 520 of a cameraapplication, where the preview interface 520 includes a preview area521. The preview area 521 is used to display the RGB skin image acquiredby the camera 193.

Step 302. The processor 110 extracts a pigment from an R channel, a Bchannel, and a G channel of the first image based on a correspondencebetween a first spectral response curve of the pigment and a secondspectral response curve of the device having the RGB image photographingfunction.

The pigment may include but is not limited to any one of the following:hemoglobin, melanin, carotene, lipochrome, and bile pigment.

Specifically, the processor 110 extracts the pigment from the R channel,the B channel, and the G channel of the first image based on thecorrespondence between the first spectral response curve of the pigmentand the second spectral response curve of the device having the RGBimage photographing function, and channel values of the three channels,namely the R channel, the G channel, and the B channel, of the firstimage.

For example, the first spectral response curve of the pigment reflectsabsorption values of the pigment in spectral segments of differentwavelengths, for example, an absorption spectrum curve of melanin, anabsorption spectrum curve of oxyhemoglobin, and an absorption spectrumcurve of deoxyhemoglobin that are shown in FIG. 6 . The second spectralresponse curve of the device having the RGB image photographing functionreflects absorption values, corresponding to different spectralsegments, of different photosensitive units in the device having the RGBimage photographing function, for example, an absorption spectrum curveof a red (R) photosensitive unit, an absorption spectrum curve of agreen (G) photosensitive unit, and an absorption spectrum curve of ablue (B) photosensitive unit that are shown in FIG. 6 .

In a specific example, melanin and hemoglobin are extracted from thefirst image. Because the melanin and the hemoglobin have specificabsorption spectra, the melanin and the hemoglobin can be separated fromthe body reflection component of the first image by using a spectrumanalysis technology. The following provides a possible implementation ofseparating the melanin and the hemoglobin.

Based on the first spectral response curve of the melanin shown in FIG.6 and the second spectral response curve of the device having the RGBimage photographing function, a first function relationship between thefirst spectral response curve of the melanin and the second spectralresponse curve of the device having the RGB image photographing functionmay be determined. Then, based on channel values of an R channel, a Gchannel, and a B channel of the device having the RGB imagephotographing function and the first function relationship, acorrespondence between the melanin and the channel values of the threechannels, namely the R channel, the G channel, and the B channel, may bedetermined. For example, the correspondence is represented by thefollowing formula (1):M=g(R,G,B)  (1)

In the foregoing formula (1), R, G, and B respectively represent channelvalues of three channels, namely an R channel, a G channel, and a Bchannel, of an RGB image, and M is a melanin value of the melanin, g isa mapping function from the channel values of the three channels, namelythe R channel, the G channel, and the B channel, to the melanin value M.

Based on the foregoing formula (1) and a first image extracted from anRGB skin image displayed in the preview area 521 shown in FIG. 5C, themelanin may be extracted from the first image based on channel values ofthree channels, namely an R channel, a G channel, and a B channel, ofthe first image. In this way, a grayscale image corresponding to themelanin shown in FIG. 7 can be obtained.

Similarly, based on the first spectral response curve of the hemoglobinshown in FIG. 6 and the second spectral response curve of the devicehaving the RGB image photographing function, a second functionrelationship between the first spectral response curve of the hemoglobinand the second spectral response curve of the device having the RGBimage photographing function may be determined. Then, based on thechannel values of the R channel, the G channel, and the B channel of thedevice having the RGB image photographing function and the secondfunction relationship, a correspondence between the hemoglobin and thechannel values of the three channels, namely the R channel, the Gchannel, and the B channel, may be determined. For example, thecorrespondence is represented by the following formula (2):H=f(R,G,B)  (2)

In the foregoing formula (2), R, G, and B respectively represent channelvalues of three channels, namely an R channel, a G channel, and a Bchannel, of an RGB image, and H is a hemoglobin value of the hemoglobin,and f is a mapping function from the channel values of the threechannels, namely the R channel, the G channel, and the B channel, to thehemoglobin value H.

Based on the foregoing formula (2) and the first image extracted fromthe RGB skin image displayed in the preview area 521 shown in FIG. 5C,the hemoglobin may be extracted from the first image based on thechannel values of the three channels, namely the R channel, the Gchannel, and the B channel, of the first image. In this way, a grayscaleimage corresponding to the hemoglobin shown in FIG. 8 can be obtained.

It should be noted that the foregoing formula (1) and formula (2) may bemathematical models provided in existing research, or may be modelsobtained through training by using a machine learning algorithm.

Step 303. The processor 110 generates a pseudo-color image based on theextracted pigment, and displays the pseudo-color image.

In a specific example, the processor 110 generates a grayscale image (amelanin result image shown in FIG. 7 and a hemoglobin result image shownin FIG. 8 ) based on the extracted pigment, and then converts thegrayscale image into a pseudo-color image. In this way, an intuitivepigment detection result can be presented to the user, so that the usercan obtain more information from the pseudo-color image.

Specifically, the pseudo-color image may be obtained through mapping byusing a preset color lookup table. For example, the color lookup tablemay include a mapping relationship between different grayscale valuesand R values, G values, and B values. In this way, a pseudo-color imagecan be obtained by searching for an R value, a G value, and a B valuethat are corresponding to a grayscale value of each pixel in thegrayscale image.

Further, to obtain a more accurate pigment detection result, theprocessor 110 may perform post-processing such as normalizationprocessing and contrast enhancement processing on the generatedgrayscale image, and then convert the post-processed grayscale imageinto the pseudo-color image.

According to the foregoing solution, the processor 110 extracts thefirst image from the to-be-detected RGB skin image, where the firstimage is used to represent the body reflection component in the RGB skinimage, and the RGB skin image is photographed by the device having theRGB image photographing function; and the processor 110 extracts thepigment from the first image based on the relationship between the firstspectral response curve of the pigment and the second spectral responsecurve of the device having the RGB image photographing function. In thisway, the processor 110 extracts the pigment based on the spectralresponse relationship, so that pigments in RGB skin images photographedin different scenarios can be detected, avoiding a case in the prior artin which pigment detection can be performed only after training isperformed in advance by using skin images acquired in differentscenarios. Therefore, pigment detection based on this solution hasrelatively good applicability.

Based on the foregoing embodiment, to improve fidelity of the firstimage extracted from the RGB skin image, the following provides anoptional implementation, and step 301 is implemented by using steps S1to S3 below.

S1. The processor 110 converts the to-be-detected RGB skin image into afirst Lab image.

Specifically, the processor 110 converts the to-be-detected RGB skinimage from RGB color space into Lab color space, to obtain the first Labimage. The conversion from the RGB color space to the Lab color spacecan be performed by using an algorithm disclosed in the industry.

The RGB color space includes three channels, namely an R channel, a Gchannel, and a B channel. A color of each pixel in the to-be-detectedRGB skin image includes an R value, a G value, and a B value, R valuesof all the pixels form the R channel, G values of all the pixels formthe G channel, and B values of all the pixels form the B channel.

The lab color space includes three channels, namely an L channel, an achannel, and a b channel. L represents pixel luminance, and is unrelatedto color information. a and b are related to a color of a pixel, and areunrelated to the pixel luminance. a represents a range from magenta togreen, and b represents a range from yellow to blue. A color of eachpixel in the first Lab image includes an L value, an a value, and a bvalue. L values of all the pixels in the first Lab image form the Lchannel, a values of all the pixels form the a channel, and b values ofall the pixels form the b channel.

If filtering processing is performed on the R channel, the G channel,and the B channel of the RGB skin image, color overlapping may occur ina filtering result. As a result, fidelity of an obtained image is poor.The to-be-detected RGB skin image is converted into the first Lab image,so that color-related information and color-unrelated information can beseparated, to separately process the color-related information in thefirst Lab image.

S2. The processor 110 extracts a body reflection component from each ofthe L channel, the a channel, and the b channel of the first Lab image.

For example, the processor 110 may separately perform filteringprocessing on the L channel, the a channel, and the b channel of thefirst Lab image, to obtain the body reflection components of the Lchannel, the a channel, and the b channel of the first Lab image.

An initial value of each of the L channel, the a channel, and the bchannel of the first Lab image includes a body reflection component anda surface reflection component. In an implementation, filteringprocessing is separately performed on the initial values of the Lchannel, the a channel, and the b channel of the first Lab image toobtain surface reflection components of the L channel, the a channel,and the b channel of the first Lab image, and then body reflectioncomponents of the L channel, the a channel, and the b channel aredetermined. The body reflection component of the L channel of the firstLab image is a difference between an initial value of the L channel ofthe first Lab image and a surface reflection component of the L channelof the first Lab image; the body reflection component of the a channelof the first Lab image is a difference between an initial value of the achannel of the first Lab image and a surface reflection component of thea channel of the first Lab image; and the body reflection component ofthe b channel of the first Lab image is a difference between an initialvalue of the b channel of the first Lab image and a surface reflectioncomponent of the b channel of the first Lab image. In this way, thesurface reflection components of the L channel, the a channel, and the bchannel may be filtered out, to accurately extract the body reflectioncomponents from the L channel, the a channel, and the b channel.

Further, to improve pigment detection accuracy, the surface reflectioncomponents of all the L channel, the a channel, and the b channel of thefirst Lab image are obtained by separately performing bilateralfiltering processing on the initial values of the L channel, the achannel, and the b channel of the first Lab image. This solution canhelp further retain edge information of the first image, therebyimproving the pigment detection accuracy.

S3. The processor 110 combines the body reflection components extractedfrom the L channel, the a channel, and the b channel of the first Labimage to obtain a second Lab image.

S4. The processor 110 converts the second Lab image into an RGB image toobtain the first image.

Specifically, the processor 110 converts the second Lab image from theLab color space into the RGB color space, to obtain the first image. Theconversion from the Lab color space to the RGB color space can beperformed by using an algorithm disclosed in the industry.

According to steps S1 to S4, the color-related information and thecolor-unrelated information can be separately processed, thereby helpingimprove the fidelity of the first image.

It should be understood that the foregoing embodiments of thisapplication may be used in combination, or may be separately used.

In the foregoing embodiments provided in this application, the methodprovided in the embodiments of this application is described from aperspective of the electronic device serving as an execution body. Toimplement the functions in the method provided in the embodiments ofthis application, the electronic device may include a hardware structureand/or a software module, and implement the functions in a form of thehardware structure, the software module, or a combination of thehardware structure and the software module. Whether a function in theforegoing functions is performed by the hardware structure, the softwaremodule, or the combination of the hardware structure and the softwaremodule depends on a particular application and a design constraint ofthe technical solution.

Based on a same concept, FIG. 9 shows an electronic device 900 accordingto this application. For example, the electronic device 900 includes atleast one processor 910, a memory 920, and a display screen 930, and mayfurther include the display screen 930 and a camera 940. The processor910 is coupled to the memory 920, the display screen 930, and the camera940. The coupling in this embodiment of this application is an indirectcoupling or a communication connection between apparatuses, units, ormodules, may be in an electrical form, a mechanical form, or anotherform, and is used for information exchange between the apparatuses,units, or modules.

Specifically, the memory 920 is configured to store a programinstruction.

The processor 910 is configured to invoke the program instruction storedin the memory 920, so that the electronic device 900 performs the stepsperformed by the electronic device in the pigment detection method shownin FIG. 3 .

The display screen 930 is configured to display a pigment detectionresult obtained by the processor 910, for example, display thepseudo-color image in step 303. The display screen may further beconfigured to display a preview interface when the camera 940 startsphotographing, where the preview interface includes an image acquired bythe camera 940, and is used to display the user interface designed inthe foregoing embodiment.

It should be understood that the electronic device 900 may be configuredto implement the pigment detection method shown in FIG. 3 in theembodiments of this application. For related features, refer to theforegoing descriptions. Details are not described herein again.

A person skilled in the art may clearly know that the embodiments ofthis application may be implemented by using hardware, firmware, or acombination thereof. When the present invention is implemented bysoftware, the foregoing functions may be stored in a computer-readablemedium or transmitted as one or more instructions or code in thecomputer-readable medium. The computer-readable medium includes acomputer storage medium and a communications medium, where thecommunications medium includes any medium that enables a computerprogram to be transmitted from one place to another. The storage mediummay be any available medium accessible to a computer. By way of exampleand not limitation, a computer-readable medium may include a RAM, a ROM,an electrically erasable programmable read only memory (EEPROM), acompact disc read-only memory (CD-ROM) or another optical disc storage,a magnetic disk storage medium or another magnetic storage device, orany other computer-accessible medium that can be used to carry or storeexpected program code in an instruction or data structure form. Inaddition, any connection may be appropriately defined as acomputer-readable medium. For example, if software is transmitted from awebsite, a server or another remote source by using a coaxial cable, anoptical fiber/cable, a twisted pair, a digital subscriber line (DSL) orwireless technologies such as infrared ray, radio and microwave, thecoaxial cable, optical fiber/cable, twisted pair, DSL or wirelesstechnologies such as infrared ray, radio and microwave are included infixation of a medium to which they belong. A disk and disc used by theembodiments of this application includes a compact disc (CD), a laserdisc, an optical disc, a digital video disc (DVD), a floppy disk and aBlu-ray disc, where the disk generally copies data by a magnetic means,and the disc copies data optically by a laser means. The foregoingcombination should also be included in the protection scope of thecomputer-readable medium.

In summary, what is described above is merely embodiments of thisapplication, but is not intended to limit the protection scope of thisapplication. Any modification, equivalent replacement, improvement, orthe like made according to the disclosure of this application shall fallwithin the protection scope of this application.

What is claimed is:
 1. A pigment detection method, comprising:extracting a first image from an RGB skin image, wherein the first imageis used to represent a body reflection component in the RGB skin image,and the RGB skin image is photographed by a device having an RGB imagephotography function; extracting a pigment from an R channel, a Bchannel, and a G channel of the first image by determining a functionrelationship between a first spectral response curve of the pigment anda second spectral response curve of photosensitive units in the device,and by determining a correspondence between the pigment and channelvalues of the R channel, G channel, and B channel; generating apseudo-color image based on the extracted pigment; and displaying thepseudo-color image.
 2. The method of claim 1, wherein the method furthercomprises: converting the RGB skin image into a first Lab image;extracting a body reflection component from each of an L channel, an achannel, and a b channel of the first Lab image; combining the bodyreflection components extracted from the L channel, the a channel, andthe b channel of the first Lab image to obtain a second Lab image; andconverting the second Lab image into an RGB image to obtain the firstimage.
 3. The method of claim 2, wherein the body reflection componentof the L channel of the first Lab image is a difference between aninitial value of the L channel of the first Lab image and a surfacereflection component of the L channel of the first Lab image; the bodyreflection component of the a channel of the first Lab image is adifference between an initial value of the a channel of the first Labimage and a surface reflection component of the a channel of the firstLab image; and the body reflection component of the b channel of thefirst Lab image is a difference between an initial value of the bchannel of the first Lab image and a surface reflection component of theb channel of the first Lab image, wherein the surface reflectioncomponents of the L channel, the a channel, and the b channel of thefirst Lab image are obtained by separately performing filteringprocessing on the initial values of the L channel, the a channel, andthe b channel of the first Lab image.
 4. The method of claim 3, whereinthe surface reflection components of the L channel, the a channel, andthe b channel of the first Lab image are obtained by performingbilateral filtering processing on the initial values of the L channel,the a channel, and the b channel of the first Lab image.
 5. The methodof claim 1, wherein the pigment comprises hemoglobin.
 6. The method ofclaim 1, wherein the pigment comprises melanin.
 7. The method of claim1, wherein the pigment comprises carotene.
 8. The method of claim 1,wherein the pigment comprises lipochrome.
 9. The method of claim 1,wherein the pigment comprises bile pigment.
 10. An electronic device,comprising: a memory comprising instructions; and a processor coupled tothe memory, the instructions being executed by the processor to causethe electronic device to: extract a first image from an RGB skin image,wherein the first image is used to represent a body reflection componentin the RGB skin image, and the RGB skin image is photographed by adevice having an RGB image photography function; extract a pigment froman R channel, a B channel, and a G channel of the first image bydetermining a function relationship between a first spectral responsecurve of the pigment and a second spectral response curve ofphotosensitive units in the device, and by determining a correspondencebetween the pigment and channel values of the R channel, G channel, andB channel; generate a pseudo-color image based on the extracted pigment;and display the pseudo-color image.
 11. The electronic device of claim10, wherein the instructions further cause the electronic device to:convert the RGB skin image into a first Lab image; extract a bodyreflection component from each of an L channel, an a channel, and a bchannel of the first Lab image; combine the body reflection componentsextracted from the L channel, the a channel, and the b channel of thefirst Lab image to obtain a second Lab image; and convert the second Labimage into an RGB image to obtain the first image.
 12. The electronicdevice of claim 11, wherein the body reflection component of the Lchannel of the first Lab image is a difference between an initial valueof the L channel of the first Lab image and a surface reflectioncomponent of the L channel of the first Lab image; the body reflectioncomponent of the a channel of the first Lab image is a differencebetween an initial value of the a channel of the first Lab image and asurface reflection component of the a channel of the first Lab image;and the body reflection component of the b channel of the first Labimage is a difference between an initial value of the b channel of thefirst Lab image and a surface reflection component of the b channel ofthe first Lab image.
 13. The electronic device of claim 12, wherein thesurface reflection components of the L channel, the a channel, and the bchannel of the first Lab image are obtained by separately performingfiltering processing on the initial values of the L channel, the achannel, and the b channel of the first Lab image.
 14. The electronicdevice of claim 12, wherein the surface reflection components of the Lchannel, the a channel, and the b channel of the first Lab image areobtained by performing bilateral filtering processing on the initialvalues of the L channel, the a channel, and the b channel of the firstLab image.
 15. A computer program product for detecting a pigment, thecomputer program product being embodied in a non-transitory computerreadable medium and comprising computer instructions for: extracting afirst image from an RGB skin image, wherein the first image is used torepresent a body reflection component in the RGB skin image, and the RGBskin image is photographed by a device having an RGB image photographyfunction; extracting a pigment from an R channel, a B channel, and a Gchannel of the first image by determining a function relationshipbetween a first spectral response curve of the pigment and a secondspectral response curve of photosensitive units in the device, and bydetermining a correspondence between the pigment and channel values ofthe R channel, G channel, and B channel; generating a pseudo-color imagebased on the extracted pigment; and displaying the pseudo-color image.16. The computer program product of claim 15, wherein the computerprogram product further comprises computer instructions for: convertingthe RGB skin image into a first Lab image; extracting a body reflectioncomponent from each of an L channel, an a channel, and a b channel ofthe first Lab image; combining the body reflection components extractedfrom the L channel, the a channel, and the b channel of the first Labimage to obtain a second Lab image; and converting the second Lab imageinto an RGB image to obtain the first image.
 17. The computer programproduct of claim 16, wherein the body reflection component of the Lchannel of the first Lab image is a difference between an initial valueof the L channel of the first Lab image and a surface reflectioncomponent of the L channel of the first Lab image; the body reflectioncomponent of the a channel of the first Lab image is a differencebetween an initial value of the a channel of the first Lab image and asurface reflection component of the a channel of the first Lab image;and the body reflection component of the b channel of the first Labimage is a difference between an initial value of the b channel of thefirst Lab image and a surface reflection component of the b channel ofthe first Lab image.
 18. The computer program product of claim 17,wherein the surface reflection components of the L channel, the achannel, and the b channel of the first Lab image are obtained byseparately performing filtering processing on the initial values of theL channel, the a channel, and the b channel of the first Lab image. 19.The computer program product of claim 17, wherein the surface reflectioncomponents of the L channel, the a channel, and the b channel of thefirst Lab image are obtained by performing bilateral filteringprocessing on the initial values of the L channel, the a channel, andthe b channel of the first Lab image.
 20. The computer program productof claim 15, wherein the pigment comprises hemoglobin.